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ABSTRACT 

The Business Communication courseware is a supplementary learning tool used in Universiti Tun Abdul Razak. It is used to 
complement face to face classes, online tutorials and lecture notes. The courseware's Instructional Design has been 
summatively evaluated with positive results. To ensure guality, the screen design is evaluated as well. Students' evaluations 
of the screen design of the courseware are gathered and analyzed. The results of the research show that the courseware's 
screen design is favoured and effective as a supplementary material in the university. The outcome of the summative 
evaluation will be used to improve the guality of teaching as well as the development of other courseware or instructional 
materials in the future. 


INTRODUCTION 

Universiti Tun Abdul Razak, UNITAR is one of the leading 
virtual universities in Malaysia. It started its operation on the 
18 th of December 1997. It is an English medium, virtual 
university which combines the use of Online Tutorials (OLT), 
face to face interaction and multimedia environment 
specifically, Interactive Multimedia courseware. The 
philosophy of the university is to develop a renaissance in 
the goals, content and method of teaching and learning 
(Unitar, 1996). 

Education per semester here is through weekly face to 
face classes (2 hours per class), OLTs once a fortnight (2 
hours per session) and the use of courseware as 
supplementary materials. Other channels needed to 
communicate with the educators are via email, fax or 
telephone. The objective of the university is to provide 
quality education for any person, of any age and from 
anywhere, via virtual education, hence, there motto is 
'Through Quality We Educate 1 (Unitar, 1996). 

Investments of millions have been made by the university 


to create and design courseware for every courses taught. 
Business Communication is a compulsory course for all 
students at the university as it is a language course. It is a 
requirement for all students to have certain prerequisites 
before going through the course. They need to have at 
least a C6 or the New Code 6C from the Malaysian 
Education Certificate (SPM) English examinations or a 
Band 2 for the Malaysian University English TEST (MUET). 

After three years of implementation, this is the first time that 
the courseware is being summatively evaluated for its 
screen design. Instructional materials need to be 
evaluated afterthey have been produced. This is to ensure 
that they are effective instructional tools. Hence, there is a 
need to take the fifth step of the ADDIE model here. The 
ADDIE model is a 'generic model that includes the 
processes of Analysis, Design, Development, 
Implementation and Evaluation' (Gibbons, 1981) The 
model is created based on the fact that learning or the 
creation of materials should happen in a planned and 
orderly manner. 
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The study is a general assessment of the overall product 
which is the courseware. Students are asked to rate the 
helpfulness of specific course features to identify and 
improve the quality of the courseware. The focus is on the 
screen design and not on the learners, even though they 
are the ones who judge what they like or do not like and 
suggest ways to improve the courseware. 

Statement of the Problem 

All courses offered in Unitar are divided into three 
components. They are the face to face classes, online 
tutorials and courseware as a supplementary tool. The aim 
is to ensure that the students of the virtual university will 
achieve mastery of knowledge and skills effectively. Here, 
the focus is on the students. There is no barrier in time or 
space. For example, for the online tutorials, students do not 
have to be physically present in Unitar to attend a class. 

All students are given a courseware in the form of a CD 
ROM. This works as a supplementary material for the 
course. The idea of using the courseware is thoroughly 
applaudible as it is considered to be a helpful Interactive 
Multimedia tool that may help the students to understand 
the course better especially when they are on their own at 
home. 

The tool, just as any other instructional tool has gone 
through the Analysis, Design, Development and 
Implementation stages but the last and equally important 
stage which is the Evaluation Stage has not been 
administered. No summative evaluation research or study 
on the screen design has never been done on the 
courseware. As all tools need to go through the Evaluation 
stage to determine its effectiveness, this study is vital. 
Purpose of the Study 

The purpose of this research is to summatively evaluate the 
screen design of a Business Communications courseware. 
The courseware is specifically known as The Fundamentals 


of Business Communications 1. It consists of additional 
information of the subject and quizzes to complement the 
Business Communication course. The steps to be taken 
are, to have students evaluate the courseware and later 
the researcher could reveal any weaknesses in its screen 
designs. The study is mainly to determine the attitude of 
students towards using the Business Communication 
courseware as a supplementary material in Unitar. 
Research Question 

This study is designed to answer the following research 
question: 

How effective is the screen design of the Business 
Communication courseware? 

Significance of Research 

In this study, a Business Communication Courseware is 
evaluated based on the guidelines from research findings 
and learning theories. The need for research to be done 
here is to analyze the effectiveness of a Business 
Communication courseware and to check if it has fulfilled 
the criteria of a good courseware. To do that, the fifth step 
of the ADDIE model needs to be taken. If the courseware 
does not meet the targeted criteria as an instructional tool, 
some faster steps are needed to be taken to enhance the 
quality of the courseware used. The criteria to be covered 
are the instructional, screen and interface designs based 
on existing literature. For this study, focus is on the screen 
design of the courseware. Here are the criteria breakdown 
Screen Design - The Text 

The Graphics 
The Layout 

(Hannafin & Hooper, 1989; Schwier & Misanchuk, 1993) 

The Screen Design 

Screen Design is the process of displaying texts, graphics 
and layout on a device with a surface made of clear glass 
or plastic and attached to a computer. Screen Design 
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looks at the text style, size and colours. The graphics is in 
terms of the menu bars, video, pictures, diagrams, 
animations and photographs used, and the layout covers 
the screen layout, background graphics and colours 
(Hannafin & Hooper, 1989; Schwier & Misanchuk, 1993). 
Summative Evaluation 

Summative Evaluation is the judging of a completed and 
not a developing teaching material, according to Kemp, 
Morrison and Ross (1994). A lot of times designers, creators 
or even users think that materials they have produced are 
satisfactorily good products. Valid support is needed to 
confirm the materials produced. 

Summative evaluation is also implemented because 
there is a need for information that will be used to base 
decisions on for adoption. It was stated that it is 'more 
credible if the design team does not summatively 
evaluate the instructional materials because they are less 
likely to be objective about what they created 1 . Here the 
objective is to diagnose 'specific instructional 
disadvantages in order to improve instructions' (Bass, 
2000 ). 

To get true answers and honest opinions on the quality of 
materials created, it is important to carry out a summative 
evaluation. Kemp, Morrison and Ross (1994) mention that 
some of the important issues that can be examined using 
summative evaluation are: 

? The effectiveness of learner or trainee learning. 

? The efficiency of learner or trainee learning. 

? The cost of programme development and continuing 
expenses in relation to effectiveness and efficiency. 

? The attitudes and reactions to the programme by 
learners, faculty and staff. 

? The long term benefits of the instructional programme. 

If deficiencies are discovered, designers, creators and 
trainers should know and improve them for the sake of 


handling the present materials in class better as well as 
improving on them for future use. 

In this study, the fourth issue by Kemp, Morrison and Ross is 
focused on. Here, the researcher looks at the effectiveness 
of a supplementary material by studying the attitudes, 
reactions and responses to the programme by the 
learners in terms of the courseware's screen design. 
Although it was stated that 'measurement of effectiveness 
can be ascertained from test scores, ratings of projects or 
performances and records of observations of learners' 
behaviour and that the current interest is on summatively 
evaluating programmes by measuring students' portfolios, 
seminars, performances and exhibits' (Seels & Glasgow, 
1998), this study will only look at the responses to the 
programme by the learners. This is because, as a 
supplementary material, the test scores, ratings of 
projects, performances, portfolios, seminars and exhibits 
do not speak of the courseware's effectiveness or the lack 
of it as a supplementary material and will not fairly 
representthe quality of the courseware's screen design. 

If 100 students' responses were evaluated, the 
effectiveness of the courseware can be determined. If 
80% of them evaluated the courseware as excellent, this 
would mean that the courseware is highly acceptable as 
an effective supplementary material and if the evaluation 
is below the acceptable level, a proposal could be made 
to the higher authority, lecturers, designers and all involved 
in the use of the courseware. The lecturers can be aware of 
the existence of certain disadvantages of the courseware 
and guide the students with some instruction on how they 
can use it. 

No doubt that in most circumstances, judgement by 
experts is one popular way of evaluating the instructional 
materials (Dick & Carey, 1996). However, Dudley-Marling & 
Owston (1990) suggested another view which the 
researcher finds refreshing. They said that'the efficacy of a 
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piece of educational software must be judged according 
to the responses of students to that programme and not 
the predictions of adults'. It seems that some experts have 
evaluated some instructional materials as excellent only to 
find that some students found the jerky and slow ones, 
excellent (Heath, 1989). The overall quality of an 
instructional material should be viewed by experts as well 
as the actual users who are the students themselves. 
Business Communication 

The courses in the English Unit of UNITAR provide a bridge 
from basic skills in Listening, Speaking, Reading and 
Writing, to transfer level courses in English for business 
communication and information technology. The Business 
Communication courses are designed with immediate 
and long-term aims in mind. The immediate aims are to 
provide the students with a conducive learning 
environment in which to acquire the cognitive skills of 
reading, thinking, and understanding texts for their core 
courses, and the productive skills for them to converse,to 
do their assignments and answer to take examination 
questions in English. 

The long-term aims are to ensure that our students develop 
the skills necessary to succeed in their careers in a world of 
continuous change, and lead lives that are meaningful, 
happy and fruitful (Unitar, 1996). 

Courseware as a Supplementary Material 
'Teachers and books are severely limited in what they can 
transmit to students out of their experience and content 
alone. Media and technology can provide students with 
effective means of learning from vicarious experiences' 
(Edgar, 1954). 

The use of courseware in higher learning institution is said to 
be the best revolution and innovation that can ever 
happen in our education system. Being an extremely 
unique supporting material for the classrooms, it caters for 


individual needs as well as changes in the whole 
environment of the classroom, instead of the 
conventional teaching and learning styles, usage of 
courseware changes the teaching system of today. 
Authentic Context I Activities 

Interactive Multimedia (IMM) courseware invites real life 
exploration and brings into the classroom the real world. 
For instance, by using a courseware, a student can 'swim' 
with the haemoglobins, leucocytes and hormones in the 
human bloodstream and understand and experience the 
advantages and improvement that take place with the 
consumption of the Paracetamol. 

Coaching and Scaffolding 

According to the Oxford Advanced Learner's Dictionary, 
coaching is, 'the teaching and training of students 
especially for an examination'. Another related concept is 
scaffolding. An expert on educational scaffolding, 
Winnips (2000) defines the process of scaffolding as 
'providing support to student learning and then retreating 
that support so that the students become self-reliant. 
Scaffolding, in the form of coaching or modeling, supports 
students as they develop new skills or learn new concepts. 
When the students achieve competence, the support is 
removed. The students, then, continue to develop the skills 
or knowledge on their own. 

Almost all IMM courseware provide a step by step 
guidance into the educational and navigational 
processes involved in dealing with tutorials or assessments. 
Being able to observe an expert's performances before 
attempting, allows the students to absorb strategies of the 
processes modeled. So, students can observe, learn and 
follow the steps portrayed by the 'expert' of the IMM 
courseware. It provides an environment that supports the 
students when in trouble, somewhere in the middle of a 
tutorial. The provision of coaching and scaffolding helps 
the students to become independent enough to stand on 
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their own. For example, when the question 'Who is the first 
Prime Minister of Malaysia 1 is asked, a student who clicks on 
Datuk Seri Dr. Mahathir Mohammad will be given 
immediate feedback that says 'Datuk Seri Dr. Mahathir 
Mohammad is the fourth Prime Minister of Malaysia'. Here, 
the student actually 'learns from his mistake'. Lajoie (1993) 
describes an environment for 'avionics troubleshooting 
entitled Sherlock 1' that may provide coaching similar to a 
human tutor's exhibition of coaching. If this happens in the 
Malaysian education, learning would go to a much higher 
and desirable stage than ever possible. 

IMM courseware caters to the need of each student's 
learning differences as well. Students get to proceed at 
their own pace in this comfortable environment and 
receive immediate feedback. Both proficient and slow 
learners can fully utilize the same IMM courseware with 
satisfaction. 

Integrated Assessment 

Assessments, short tests and quizzes of an IMM courseware 
are formative and summative. These tests are not formal. 
They are informal, interactive and students do not feel the 
pressure or stress as they would normally feel when given a 
paper and pencil test. They are given positive feedback 
that motivates them to either try again or go to a new level. 
With the IMM courseware, they learn within the tasks given 
and feel comfortable receiving feedback on the mistakes 
they make. 

Easy Information 

There is a riddle that shows the capability of an IMM 
courseware and it goes like this; 'How can one child carry 
3,00,000 pages of printed materials by himself using only 
one hand? He carries a courseware. An IMM courseware 
contain numerous tutorials and tests as compared to one 
book. It brings to education the extraordinary storage and 
delivery capabilities of computerized materials which are 
hard to obtain, but extremely important for those who find 


books and other resources from the libraries. 

Apart from that, the linkages that an IMM courseware 
provides could take students from one related idea to the 
next in ways that vary depending on the students' interest 
and background experience. The process is only done 
with a click of a button and students do not have to flip 
through to search for a lost page or topics of interest. Users 
have control over what they are experiencing and this 
builds confidence and caters to students' individual 
learner differences. 

Historical films such as Hitler's cruel research in 
'hypothermia', rare sound recordings of famous speeches 
like Malaysian's first Prime Minister, 'Tuanku Abdul Rahman's 
Declaration of Independence are easily accessible now 
with the technology existing in IMM. As we know, the 
human mind does not operate in a linear form. Provided 
that the needed information is stored, the IMM has the 
ability to allow a person to view different information side 
by side leading to the fulfillment of Malaysian Educational 
Philosophies (Jaring, 1995). 

Interactivity 

Agnew, Kellerman & Meyer (1996) said that 'students learn 
better if they construct knowledge actively and not merely 
receive information passively'. 

Students learn to think more effectively and become more 
self directed when they learn actively. Some students can 
better absorb information through a particular medium, 
for example: songs, plays, colours and pictures. IMM 
technology appeals to all senses of the humanbeings. 
When all senses are triggered, the learning that takes 
place is better. It also caters for a much wider variety of 
people. Students read, listen, look and manipulate the 
materials of their own choice actively. Thus, the attention 
span is prolonged and lengthened than normally possible 
and information is stored better in memory. 
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Good Resource For Students and Educators 
With the use of IMM courseware, teaching is enhanced 
because classroom presentations are not only about an 
educator coming in, talking and bringing in some 
materials while students listen. The whole process of 
learning changes. Instead of teaching, educators 
facilitate and guide the students towards the completion 
of their tasks. 

The productive use of IMM courseware makes teaching 
and learning more efficient and effective. Educators or 
students can refer to any IMM courseware if some 
questions arise and the information is presented for 
cognitive construction in a very engaging way. Another 
useful tool that it has is, its capability to keep student's 
records individually. Students can type in their name and 
all information pertaining to their performances will be 
displayed. Educators and even students themselves will 
get to check their own progress. 

Social Benefits 

There are always situations where one educator has too 
many students to teach i.e. 1:50 ratio. Whether we want to 
or not, we somehow would 'neglect' some students due to 
the insufficient time and energy that we have. Thus, we do 
injustice to some students. IMM courseware can solve this 
problem and erase the guilt and frustration in educators for 
not being able to give sufficient attention to all students. It 
meets all students' needs and nobody's need will be 
overlooked. Educators need only to guide the students. 

'The student thinks, the teacher guides and the computer 
facilitates' (Kaur, 1996). This opportunity can solve the burnt 
out syndrome, that most teachers suffer within the 
Malaysian education system. Dwyer (1995) reports in the 
Apple Classroom of Tomorrow (ACOT) that 'there was 
dramatic decrease in teacher-led activities and a 
corresponding increase in cooperative activities' with the 
use of IMM courseware. IMM courseware promotes 


cooperation and collaboration among students and 
educators. They help each other without being asked, 
when using the IMM courseware, hence, the inculcating of 
moral values as well. 

The Screen Design 

The Screen Design is the process of displaying texts, 
graphics and layout on a device with a surface made up 
of clear glass or plastic and attached to a computer. 'If 
properly developed and properly implemented could 
revolutionize education'. (Menn, 1993). The Screen Design 
refers to the title and text sizes, colours and 
appropriateness, as well as the functions of graphics and 
layout. 

Screen Texts 

The structures of the text cover the font style, colour, height, 
spacing and their readability. Pert (1989) named four 
factors that affect instructions and they are 'size, style, 
spacing and contrast between lettering and background'. 
Baker (2000) stated that text used in courseware should be 
clear. This is because, though the courseware may be 
'evolving significant multimedia capabilities, it is still home 
to primarily one medium which is the text'. In an instruction, 
the screen design and all the elements placed on it, 
especially the text, communicate to the students. A 
selection of fonts should be chosen and used consistently 
throughout the courseware. Two different font types should 
be used for headings and for the body of the text. 

The header text serves as the document structure and 
helps students scan the document. Well defined header 
text should stand out from the body text and should be 

1. Bold 

2. Larger than body text 

3. Serif or sans-serif; not script 

Body text which is the basis of most content, occupies a 
large part of the page. The text should not be smaller than 
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the default size even though the screen is filled with other 
graphical elements. Generally, body text should be black 
serif text on a white background or any other lighter 
colours. Georgia, Verdana, Arial or Comic Sans MS fonts 
which are large enough and clearly designed for the 
screen should be used according to their functions. Body 
text characteristics should be of default size or slightly 
larger. The screen font should be preferably serif or a well 
designed sans serif font, plain or bold or italic for emphasis 
and no more than double spacing. (Baker, 2001) 

The texts should be well structured and if all instructions are 
positioned at the bottom left of the screen, it should be 
followed through, till the end of the lessons. Above all, the 
content provided should not be too long. If there are a lot 
to be provided, keywords should be used. Otherwise the 
instructions will lose its ability to hold attention. 

The letter height should be consistent and should be 
around the area of 1/4 inch but it can still be reduced or 
added depending on the light and colour used. 

There should be a consistency and coherency in the use of 
colours for text. If the colour red, is used on the first screen, 
the concept must be followed through until the end of the 
lesson. Mixing and changing colours of texts for the sake of 
brightening the screen without thinking of the students' 
needs and the effects on the lessons, would jeopardize the 
whole instructions planned (Pert, 1989). Basically, text 
should be legible and clear for the best teaching and 
learning to take place. 

Screen Colour 

The colours used on the screen should also be based on 
principles of colours in instructions. Bright colours represent 
actions and excitement whereas darker colours present a 
more formal mood. For business purposes, blue and white 
are suitable for they present the corporate and executive 
looks to the screen. The same kind of looks should be 
maintained throughout the lesson. Visited links should also 


change colours to provide the users with a notion that they 
have been there. Red or purple is normally used for the 
visited links and blue for the unvisited links. The basic 
principle to use colours in screen design is, to use colours 
that are not too colourful or too bright. 

Screen Graphics 

The graphics used must be symbolic graphics which 
means that they have certain meanings or purposes that 
are widely understood by any user for example, a small 
graphic of a house is widely accepted as 'home 1 . Apart 
from that, the graphics should fill the screen comfortably. 
Some designers forget that graphics that are too large will 
not only take up a lot of space but also cause disturbances 
to the users' eyes. However one basic rule to follow in 
screen designing is to have a few large sized graphics 
rather than a lot of smaller ones. 

To make instructions even better, graphics can be 
enhanced with light source coming from behind or 'glow' 
to draw emphasis to a certain area. A 'halo' can even be 
positioned around a graphic to provide the same effect. If 
the use of logos is included, make sure that there are no 
words or letters in them. This will make it easier for editing or 
changing of information, if it is needed in the future (Jong 
andSarti, 1994). 

Screen Layout 

The layout must be simple and consistently presenting the 
text, graphics, instructions and feedback areas. It should 
not be too crowded and there must be a separation 
between the instructions and the texts so that users know 
where they exactly are, and what to look for at a particular 
area of the screen. Graphics should not be cluttered to 
one side and cause inbalance to the screen or spreaded 
everywhere to cause an untidiness that prevents learning 
from taking place. 

The layout must follow the humans' physiological ways of 
viewing and they are from left to right and top to bottom. 
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Human eyes instinctively move like that and so the 
designer must not try to adapt a new 'creative' style and 
change the norm to something that will disrupt learning. 
For any design, the success of an instruction lies on the 
layout outline that is prepared. This is called a 'storyboard'. 
A storyboard is the outline and steps that come with 
complete instructions for the interface, audio, video, 
screen and graphic designers as well as programmers, 
prepared by the instructional designer. It is for the 
designers to visualize and create animations or other 
features while scripting is being done especially in the 
development of media elements. Without it, the creation 
of instructions will not be possible. 

Malaysia is on the right path in starting the Smart School 
system and two virtual universities; Universiti Tun Abdul 
Razak (1997) and Multimedia University (1999). The 
blueprint entails the change in the culture and practices of 
Malaysia's education system especially in 'stimulating 
thinking, creativity and catering to individual abilities and 
learning styles'. With the implementation of such system, it 
is timely that IMM courseware is utilized in the learning 
institutions. 

Moreover, it offers a reality of experience which stimulates 
students' self activity, develop a continuity of thought, 
provide experience not easily obtained through other 
materials and contribute to the efficiency, depth and 
variety of learning and concrete basis for conceptual 
thinking. 

The importance of screen design must be of high priority in 
making courseware utilization for success. There is a need 
to learn and accept changes in education because it is 
not static but dynamic and so, keeps on changing. 

Research Methodology 

The population is concerned from the virtual university itself 
UNITAR. The population represents students pursuing a 
Bachelor's degree and using the courseware entitled 


'Business Communication'. The sample is 100 subjects 
from several 'Business Communication' classes. There are 
72 females and 28 males involved in the study. Their age 
range from 18-25 and all of them have not taken the 
course before. The principle of randomization is applied 
here as 100 students of several 'Business Communication' 
classes are chosen randomly for sampling from the 
registration. These 100 students represent the whole new 
batch of students taking the 'Business Communication' 
subject. This means none of them has ever used the 
courseware. They are given a duration of two weeks to go 
through the courseware and after the two weeks are up, 
the instrument is administered and data are collected. 
Among the variables included are students' age, gender 
and level of exposure to information technology and 
courseware. 

Instrumentation 

The instrument is built based on adaptations from The 
Regents of the University of California's 'Sample Student 
Survey 1 , Abtar Kaur's 'Expert Review of Storyboards' ( 1 996 ] 
and Educational Technology's 'Events of Instruction Used In 
Multimedia Courseware Design' (1990) for research 
purposes. It is developed by identifying components and 
criteria of the courseware based on the literature review 
done on screen design. They are then listed and 
accompanied by 5 point ordinal rating scales for students' 
evaluation. 

It lists items used to acquire the data of students' variables 
and their opinion on the courseware's screen design. The 
students are given about two weeks to go through the 
courseware before the evaluation is carried out. All the 100 
subjects are asked to fill up the instrument provided 
according to their experience in navigating through the 
courseware. The interest lies in finding out how students 
react to the courseware. 


l-manager’s Journal of Educational Technology Vol. 3 • No. 3 • October - December 2006 


55 






RESEARCH PAPERS 


Pilot Study 

A pilot study is done at the same university - UNITAR on a 
different sample. The randomly chosen sample here is 10 
different students taking the 'Business Communication 1 
subject. They have the same variables needed, follow the 
same procedures and processes of learning and are 
surrounded by the same environment and technological 
equipment and tools. The researcher deals with two stages 
in the pilot study. Stage one is observing one student 
navigating the courseware and jotting down anything that 
may help in perfecting the instrument. In stage two, the 
researcher observes the rest of the students as they go 
through the courseware and complete the instrument for 
reliability. After the evaluation is completed, a dialogue 
session is held to gather suggestions and further revise the 
instrument and make it a more valid and reliable one. The 
study is also done to check the time limit needed for the 
implementation of evaluation. 

Procedure 

The 100 students are given an instrument of 91 questions. 
They are briefed on the evaluation process, the objectives 
of the study and what they have to do. The instrument is 
divided into two sections, which are the screen design and 
courseware in general. They are encouraged to ask 
questions and discuss with the researcher on the items 
given. They are informed that whatever difficulties they 
encounter while interacting with the courseware are not 
totally a result of their own shortcomings and so they should 
be as honest as possible with the answers they choose. 
They are given two weeks to go through the courseware 
and the questionnaires are then collected. 

The analysis of this study takes into consideration learners' 
responses to the questionnaire regarding their attitudes 
toward the screen design of the courseware. 

Data Analysis 

A courseware entitled 'Business Communication' is first, 


given to the subjects. They are asked to carefully go 
through the courseware. The purpose of this is to let the 
students actually manipulate and navigate through the 
courseware and observe whether they face any difficulty 
or discomfort while handling the courseware. 

Students are then given an instrument that they have to go 
through. The instrument is given to all subjects to gather 
information and collect data on screen design after two 
weeks. Items are analyzed one by one. The instrument is 
administered to gather information on the advantages 
and disadvantages faced while going through the 
courseware and includes attitudinal scale of Strongly 
Disagree, Disagree, Undecided, Agree and Strongly 
Agree. The data is processed using Microsoft Excel and 
they are presented with tables. 

Data Presentation And Discussion 
The evaluation is structured as follows: 

STUDENT'S PROFILE 

1. Gender: 

Male 

Female 

2. Flow old are you? 

a) . 18-20 years old 

b) .20-25 years old 

c) .above25yearsold 

3. I use the English Language at home. 
aj.Yes bj.No c). Sometimes 

4. My English Language grade in the SPM examination is 

a.)Al b.)A2 c.)C3 d.)C4 e.)C5 f.) C6 g.)P7 

h.)P8 i.)F9 

5. I use the computer_ 

а. ) Always b.) Seldom c.jSometimes 

d.jHardly 

б. 1 have been using the computer for __ 
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a. Jlessthan 1 year 

b. ) 1 year 

c. ) 2 years 

d. ) 3 years 

e. ) more than 3 years 

7. I am familiar with educational software before coming 
to UNITAR. 

a.)Yes b.) No 

Screen Design 

1. There are enough items on the screen. 

2. Important information is immediately obvious. 

3. Instructions are placed consistently in the same place 
on the screen. 

4. I feel encouraged to continue reading. 

5. The screen layout helps me to remember what I read. 

6. Media used is appropriate to the 
content(sesuai). 

7. I can understand the graphics easily. 

8. The media supports the lesson. 

9. Using some media are necessary. 

10. The screen design is consistent. 

11. The colour used is appropriate. 

Menu 

12. Main menu is well designed. 

13. Sub-menu is well designed. 

14. Graphics chosen are appropriate (sesuai). 

15. Feedback is appropriate (appropriate use of sound, 
flashing). 

Font and Visual Quality 

16. Font styles are easy to read. 

17. Text font size is consistent. 

18. Text font size is not too small. 


19. Title font size is large enough. 

20. Font colour is suitable. 

21. Visual quality (colour, size, graphics, photography) is 
good. 

22. There is enough animation. 

Students Background 

The students who evaluated the courseware are 72% 
females and 28% males. Their ages range from 18-20 
years old (62%, n=62) and 20-25 years old (38%). 1 2% of 
the students use the English Language at home, 58% of 
them sometimes use it at home while another 30% do not 
use it at all. There are 88% of the students (n=88) always 
use the computers and 44% of them have been using the 
computer for more than 3 years. The rest have been using 
the computers between less than 1 to 2 years. Flowever, 
those who are not familiar with educational software 
before coming to Unitar as 58% of the students (n=58) 
choose 'no' to Item 7. That means their experience in using 
the computers is more on surfing the net or playing 
computer games. (Please refer to Tables 1 to 7) 


Item 

Subject 

Male 

Female 

1 

Gender 

n/% 

n/% 



28 

72 


Table 1. Gender 


Item 

Subject 

18-20 

20-25 

Above 25 

2 

Age 

n/% 

n/% 

n/% 



62 

38 

0 


Table 2. Age 


Item 

Subject 

Yes 

No 

Sometimes 

3 

Use of English 
Language 

n/% 

n/% 

n/% 



12 

30 

58 


Table 3. Use of English Language 
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Item 

Subject 

lto2 

3 

4 

5 

6 

7 

8 

9 

4 

English Language 
Grade in SPM 

n/% 

n/% 

n/% 

n/% 

n/% 

n/% 

n/% 

n/% 



2 

10 

8 

14 

26 

36 

4 

0 


Table 4. English Language Grade in SPM 


Item 

Subject 

Always 

Sometimes 

Seldom 

Hardly 

5 

Use of 
Computer 

n/% 

n/% 

n/% 

n/% 



88 

10 

0 

2 


Table 5. Use of Computer 


Item 

Subject 

< 1 Years 

1 Years 

2 Years 

3 Years 

3 Years< 

6 

No of Years 
using the 
computer 

n/% 

n/% 

n/% 

n/% 

n/% 



20 

12 

14 

10 

44 


Table 6. Number of years using the Computer 


Item 

Subject 

Yes 

No 

7 

Familiarity 

n/% 

n/% 



42 

58 


Table 7. Familiarity with educational software before coming to 
Unitar 


Data Analysis 

The analysis of data and the results are discussed based 
on the following research question. 

How effective is the Screen Design of the Business 
Communications Courseware? 

Definition of Screen Design 

The process of displaying texts, graphics and layout on a 
device with a surface made of clear glass or plastic and 
attached to a computer. 

To answer this research question, the Screen Design 
section of the learner evaluation questionnaire is 
analyzed. This section covers the text, graphics and layout 
based on Hannafin & Hooper (1989) and Schwier & 
Misanchuk(l 993)'s Screen Design outline. Based on the 


criteria set by them, questions about the menu, font, visual 
quality, animation and media are set and presented in 
Section 2 of the learner's evaluation questionnaire. There 
are altogether 23 items in this section and learners 
respond by circling or ticking against their choice. The 
evaluation data are presented in table forms for easy 
understanding and interpretations. 

Items 1 to 11 cover general statements about screen 
design. 

General Screen Information and Media 

Table 8 shows the results on General Screen Information 
and Media. Seven items are popularly evaluated 
favourably while half of it, four items are popularly 
evaluated unfavourably. Items 2, 3, 5, 6, 7, 10 and 11 
have about averagely 45.7% of the students agreeing to 
the statements about having important information 
immediately obvious (item 2), consistently placed 
instructions (item 3), screen layout that helps students to 
remember (item 5), media which is appropriate to the 
content (item 6), graphics that are easy to understand 
(item 7), consistent screen design (item 10) and 
appropriate colours used (item 11). 

However, on the other side of the anchor, 44% of the 
students (n=44) disagree that there are enough items on 
the screen. Item 4 has 34% (n=34) feeling discouraged to 
continue reading the courseware. 38% (n=38) disagree 
with item 8 and evaluated that the media does not 
support the lesson and finally, item 9 has 24% of the 
students (n = 24) who decide that some media used are 
unnecessary. It is quite alarming to see that half of the most 
frequently chosen choice is 'Disagree 1 . This shows that 
generally, the screen design could be improved so that 
the students feel more comfortable using the courseware 
and are encouraged to continue reading and using the 
courseware. Items 13 to 16 that cover the menu and 
graphics receive the students' favourable reactions. 
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Item 

Subject 

Strongly 

disagree 

Disagree 

Undecided 

Agree 

Strongly 

agree 



n/% 

n/% 

n/% 

n/% 

n/% 

1 

There is 
enough items 
on the screen. 

2 

44 

16 

26 

12 

2 

Important 
information is 
immediately 
obvious. 

0 

8 

32 

48 

12 

3 

Instructions are 
placed 
consistently in 
the same 
place on the 
screen. 

0 

22 

26 

44 

8 

4 

1 feel 

encouraged 
to continue 
reading. 

2 

34 

32 

24 

8 

5 

The screen 
layout helps 
me remember 
what 1 read. 

4 

12 

16 

52 

16 

6 

Media used is 
appropriate to 
the content. 

0 

10 

18 

56 

16 

7 

1 can 

understand 
the graphics 
easily. 

4 

26 

24 

34 

12 

8 

The media 
supports the 
lesson. 

4 

38 

36 

18 

4 

9 

Some media 
used are 
necessary. 

6 

34 

16 

32 

12 

10 

The screen 
design is 
consistent. 

2 

32 

26 

36 

4 

11 

The colour 
used is 
appropriate. 

2 

20 

24 

50 

4 


Table 8.General Screen Information and Media 


Menu and Graphics 

Table 9 shows the results on Menu and Graphics. There are 
54% of the students (n=54) who agree that the submenu is 
well designed and another 54 agree that the graphics are 
appropriately chosen. Another 38% however, are 
undecided whether the main menu is well designed and 
36% of the students (n=36) are undecided whether the 
feedback and its application of sound and flashing are 
appropriately used in the courseware. There is a cause for 
worry because item 12, which is about the "well designing 
of main menu for which 30% of the students (n=30) 
disagree to it. Item 15, shows the appropriateness of 


feedback, has 26% of the students disagreeing to it, Thus, 
there is a need to look deeper into the matter as the main 
menu and appropriateness of feedback that applies to 
media are important and if not done properly may disrupt 
the lessons. 


Item 

Subject 

Strongly 

disagree 

Disagree 

Undecided 

Agree 

Strongly 

agree 



n/% 

n/% 

n/% 

n/% 

n/% 

12 

Main menu is 
well designed. 

4 

30 

38 

20 

8 

13 

Sub-menu is 
well designed. 

4 

12 

30 

54 

0 

14 

Graphics 
chosen are 
appropriate. 

6 

12 

18 

54 

10 

15 

Feedback is 
appropriate. 

4 

26 

36 

26 

8 


Table 9. Menu and Graphics 
Font and Visual Quality 

Items 16 to 22 reveal students' evaluation of the Font used 
in the courseware. Table 10 shows the results collected on 
the Font and Visual Quality. There are 36% of the students 
(n=36) who strongly agree that the font styles are easy to 
read. 46% agree that text font size is consistent and 
another 46% agree that the text font size is not too small. 
44% of the students (n=44) agree that the text font size is 
large enough, another 44% agree that the font colour is 
suitable and 50% agree that the visual quality which 
includes colour, size, graphics and photography is good. 
However, 42% of the students disagree and 26% strongly 
disagree that there is enough animation in the courseware 
(item 22). Ranging from the age of 18 to 20 years, a liking 
for a lot of animation in the courseware is normally seen. 
Animation, after all receives a lot of support and 
admiration (in computer games and movies) by people 
of that age range. To enhance the courseware, the 
producer could add more animation but there is a need 
to be aware of the fact that the product is an academic 
product and should not promote only fun. 
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Item 

Subject 

Strongly 

disagree 

Disagree 

Undecided 

Agree 

Strongly 

agree 



n/% 

n/% 

n/% 

n/% 

n/% 

16 

Font styles are 
easy to read. 

2 

10 

20 

32 

36 

17 

Text font size is 
consistent. 

6 

12 

18 

46 

18 

18 

Text font size is 
large enough. 

4 

10 

22 

46 

18 

19 

Title font size is 
large enough. 

2 

20 

22 

44 

12 

20 

Font colour is 
suitable. 

2 

14 

20 

44 

20 

21 

Visual quality is 
good. 

4 

6 

32 

50 

8 

22 

There is 
enough 
animation. 

26 

42 

22 

8 

2 


Table 10. Font and Visual Quality 


Generally, the screen design, though needs minor 
improvements, is appropriate and evaluated favorably by 
the students. It therefore supports the use of the Business 
Communication courseware as a supplementary 
material. 

Summary 

Apart from some minor needs for improvements which 
have been identified in the respective sections, the 
instructional design, is evaluated as effective in the 
Business Communication courseware as a supplementary 
material. 

Limitations of the Study 

There are two limitations to this study. Firstly, the evaluation 
is entirely the effort of the students. This is not an ideal 
approach. The ideal approach would be to have the 
students, educators (Heath, 1981) and experts (Dick and 
Carey, 1996) evaluate the courseware. The data 
collected from only the experts are not enough because 
the students and educators are the actual users and 
would provide better feedback about the courseware. 
Hence, there is a need to have three phases of 
evaluation. 

Another limitation of this study is that the study only involves 
the students from the Kelana Jaya Study Centre and not 


the other Unitar satellite campuses all over Malaysia. Thus, 
it is not possible to infer that students from other study 
centres will provide the same outcome of results after 
using the courseware. 

Implications for further research 

'Summative evaluation was conceptualized with the 
particular purpose of aiding educators in determining 
whether ongoing instruction is worthy to be continued and 
whether available instruction is worthy of adoption 1 (Briggs, 
Gustafson and Tillman, 1991). 

The Business Communication courseware is created as a 
supplementary material in Unitar. It is an alternative 
teaching and learning tool that is used alongside with the 
face to face classes and online tutorials. The main 
objectives of the study are to summatively evaluate the 
courseware and to use the findings to improve future 
courseware or instructional materials. The findings of this 
study show that the Business Communications 
supplementary material possesses many positive 
instructional features. Nevertheless, there are certain areas 
of weaknesses that can be improved on the basis of the 
students' evaluations on the following items: 

Screen Design Strengths 
General Screen Information and Media 
Important information as immediately obvious. 
Instructions are placed consistently in the same place 
on the screen. 

The screen layout helps to remember what is read 
earlier. 

Media used is appropriate to the content. 

The students can understand the graphics easily. 

The screen design is consistent. 

The colour used is appropriate. 

Menu and Graphics 
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Submenu is well designed. 

Graphics chosen are appropriate. 

Font and Visual Quality 

Font styles are easy to read. 

Text font size is consistent. 

Text font size is large enough. 

Title font size is large enough. 

Font colour is suitable. 

Visual quality is good. 

Screen Design Weaknesses 

General Screen Information and Media 

There are not enough items on the screen. 

The students feel discouraged to continue reading. 

The lesson does not have enough support of the 

media. 

Some media used are unnecessary. 

Menu and Graphics weaknesses 

Main menu is not well designed. 

Feedback is inappropriate. 

Font 

There is not enough animation 
Quite a number of students felt that improvements should 
be made to the courseware to gain their attention. There 
are 44% students who want more items on the screen, 
34% feel discouraged to continue reading, 38% feel that 
the media does not support the lesson and 24% feel that 
some media used is unnecessary. Most students want a lot 
more animation and thus, even though the researcher 
admits that the screen design of the courseware is 
evaluated as effective, there is a need for some necessary 
minor improvements to be made. The producer should 
think about improving on the screen items and media 


used. Perhaps an extensive research could be continued 
from here to cater for at least 99% if not, 100% of the 
students' needs. This would not only improve the students' 
motivation but also the general teaching and learning 
skills of all users. 

Recommendations 

From the findings, the Business Communication 
courseware has many effective features of screen design. 
Nevertheless, there are minor weaknesses that can be 
improved upon. The minor weaknesses do not actually 
affect the learning of the course in major ways and thus, 
given proper guidance, the educators can use the 
courseware as an effective supplementary material. 
Conclusion 

The study shows the importance of summative evaluation 
of the screen design of instructional materials especially 
the Business Communication courseware used in Unitar. 
Even though the courseware was developed following 
theoretical steps, principles and procedures, there are 
minor weaknesses that are discovered. It is evident from 
this study that learners' responses to the courseware as a 
supplementary material, indicate overwhelming positive 
ratings. Findings from this study, if combined with other 
studies will help to support the use of courseware as a 
supplementary material in the university. 

Since this evaluation is done at the students' phase, there 
should be more research done at the lecturers' phase and 
finally the experts' phase for expert judgement. (Dick and 
Carey, 1991) 

Generally, the screen design is effectively used in the 
Business Communication courseware as a supplementary 
material. 
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